Face Recognition Algorithm¶

This guide walks through building a desktop GUI face-recognition project from scratch on Quectel Pi M2 Debian, using the face-recognition algorithm SDK. When launched, the application opens a window on the device desktop and supports full-screen mode.

The interface uses a three-column layout:

Left column: Algorithm modules and capabilities (face detection / 21-point landmarks / head pose / 1:N recognition)
Center column: Title, live display, and controls
Right column: Device status (face database count / RAM / ROM / CPU utilization)

Two input modes are available, and faces can also be registered at runtime:

Open Camera: Open a USB camera, detect faces in real time, and compare them against the database using 1:N matching.
Local Image: Select a local image for offline face detection and recognition.
Register Face: Enter a name to save the largest face in the current frame that has code == FACE_OK and a valid feature to facedb/registered/, then add it to the in-memory database. It is restored automatically on the next launch. Registration is not reported as successful if saving fails.

Algorithm Capabilities¶

Item	Description
Algorithm type	Face detection + 21-point landmarks + head pose + feature extraction and recognition
Detection model	facedet_512_q_3576_enc.rknn (512² input, multi-scale face boxes)
Landmark model	facelandmark_q_3576_enc.rknn (dynamic 21-point landmarks)
Recognition model	facereg_l_q_3576_enc.rknn (512-dimensional face features)
Input	Contiguous BGR memory (cv::Mat.data, rows × columns × 3)
Output	One FaceResults_t per face (xywh box + confidence + yaw/pitch/roll + 21 landmarks + feature vector)
Matching	1:N cosine-similarity matching with a default threshold of 0.6; supports up to 10,000 database entries

How results are presented in the demo:

Face boxes are green for a database match and orange for an unmatched or unusable face. The UI does not display the detection confidence conf separately.
Toggle the 21 landmarks with the [Landmarks] button.
Head pose is shown below the box as Y:yaw P:pitch R:roll.
The name and match score appear above the box. Unmatched faces are labeled Unknown; faces with code != FACE_OK are labeled low-quality. low-quality is a generic demo label; consult the SDK status code for the specific reason.

Prerequisites¶

Obtain the Resources¶

Name	Description
face_RK3576 (algorithm SDK package)	Face-recognition package containing the public headers, shared libraries, third-party runtimes, models, sample images, and face database
face_project (demo project)	Example project from this guide, including source code, Makefile, and three scripts for copying dependencies, building, and running

Key directories in the algorithm SDK package (RK_FR_NPU_v1.1/rknn-2.3.2-linux, referred to below as face_RK3576):

rknn-2.3.2-linux/
├── include/
│   ├── faceEngine.h             # Public algorithm header (the only algorithm header required for integration)
│   ├── authorization.h          # Authorization definitions (module table / error codes)
│   ├── opencv2/                 # Bundled OpenCV headers
│   └── rknn_api.h               # RKNN API header (other RKNN headers are in the same directory)
├── lib/                         # libfacerecognition.so and dependencies + libopencv_world.so
├── model/rknn/RK3576/           # Detection / landmark / recognition .rknn models
├── facedb/                      # Face database (filenames are names)
├── image/                       # Sample images
└── main.cpp                     # Official minimal usage example

Hardware and System Requirements¶

Item	Requirement
Main board	Quectel Pi M2
System	Debian / Ubuntu aarch64 with a desktop environment (Wayland or X11)
Peripheral	USB camera
Connection	The device can be accessed from a PC using adb shell

Software Requirements¶

The demo is built natively on Quectel Pi M2. Make sure the following software is installed:

g++ --version          # C++17 compiler
make --version
pkg-config --exists Qt5Widgets Qt5Gui Qt5Core && echo "Qt5 OK"   # Qt5 (LGPL) for display
v4l2-ctl --version     # Enumerate and troubleshoot USB camera nodes

UI implementation: The display layer uses the system-provided Qt5, so it displays directly on the desktop and supports full-screen mode. Capture, inference, and drawing use the OpenCV world library bundled with the SDK (libopencv_world.so, matched to libfacerecognition.so), isolated from the system OpenCV to avoid version conflicts.

Project Structure¶

face_project/
├── src/
│   └── face_cam_demo.cpp        # Main demo (Qt UI + inference worker thread + 1:N recognition)
├── include/                     # Populated by setup: SDK public header faceEngine.h + bundled opencv2/rknn headers
├── lib/                         # Populated by setup: libfacerecognition.so and dependencies + libopencv_world.so
├── models/                      # Populated by setup: detection / landmark / recognition .rknn models
├── facedb/                      # Populated by setup: face database (filenames are names)
├── images/                      # Populated by setup: sample images
├── Makefile                     # Build rules
├── build.sh                     # One-command build
├── setup_from_sdk.sh            # Copy dependencies from the SDK package into this project
├── run.sh                       # Configure environment variables and launch
└── README.md                    # Documentation

include/, lib/, models/, facedb/, and images/ are generated by setup_from_sdk.sh from the original SDK package.

Core Code Walkthrough¶

The demo uses the FaceEngine class exposed by the SDK’s public faceEngine.h header. Its lifecycle is: construct → init (load models and authorize) → set_param (set thresholds) → inference (infer each frame) → cosine_similarity (1:N matching) → uinit (release resources).

The source file is src/face_cam_demo.cpp. The following sections focus on the face-recognition calls. These snippets are from inside a function and require headers such as faceEngine.h, <cstdio>, and <cstdlib>.

Initialization¶

FaceEngine engine;
char det_model[]  = "models/facedet_512_q_3576_enc.rknn";   // Face detection
char land_model[] = "models/facelandmark_q_3576_enc.rknn";  // 21-point landmarks
char reg_model[]  = "models/facereg_l_q_3576_enc.rknn";     // 512-dimensional feature extraction
char license[512] = "face_license.ini";
if (const char* path = std::getenv("FACE_LICENSE_FILE")) {
    if (path[0]) std::snprintf(license, sizeof(license), "%s", path);
}
int module = 3;
if (const char* value = std::getenv("FACE_MODULE")) {
    if (value[0]) module = std::atoi(value);
}
int app_id = 145;
if (const char* value = std::getenv("FACE_APP_ID")) {
    if (value[0]) app_id = std::atoi(value);
}
char auth_buf[512] = {0};
if (const char* auth = std::getenv("FACE_AUTH_CODE")) {
    if (auth[0]) std::snprintf(auth_buf, sizeof(auth_buf), "%s", auth);
}
char* p_auth = auth_buf;
FACE_RES_CODE rc = engine.init(det_model, land_model, reg_model, license, module, p_auth, app_id);
engine.set_param(0.35f, 0.45f);
int alg_status = engine.get_status(0);
int auth_status = engine.get_status(1);
std::printf("rc=%d alg_status=%d auth_status=%d\n", rc, alg_status, auth_status);

This snippet demonstrates environment-variable configuration. The demo executable also accepts the authorization code as its second command-line argument, which takes precedence over FACE_AUTH_CODE.

Parameter descriptions:

init(det_model, land_model, reg_model, license, module, p_auth, app_id) loads all three models and performs authorization:
- det_model: Face detection model (512²; outputs face boxes + confidence).
- land_model: Landmark model (dynamic 21-point landmarks; also provides head pose).
- reg_model: Feature model (outputs 512-dimensional face feature vectors for matching).
- license: Path where the license file is stored. It is written after the first successful online authorization and can be used offline afterward.
- module: Platform identifier; **pass3**for RK3576 (overridable with FACE_MODULE).
- p_auth / app_id: Algorithm authorization code and authorization ID.
set_param(det_conf, nms): det_conf is the face-detection confidence threshold (default 0.35; increase it to reduce false positives); nms is the detection-box NMS IoU threshold (default 0.45).
get_status(opt) queries status: get_status(0) returns algorithm initialization status (0 means success); get_status(1) returns authorization status (0 means fully authorized; other values are defined in AuthErrCode_e).

Inference¶

Call inference once per frame (from either the camera or a local image). The SDK performs detection → landmarks → pose → feature extraction and returns a complete result for each face.

std::vector<FaceResults_t> results;
// Arguments: BGR data pointer, rows (height), columns (width), channels (must be 3), output result vector
FACE_RES_CODE inference_rc = engine.inference(
    frame.data, frame.rows, frame.cols, frame.channels(), results);
if (inference_rc != FACE_OK) {
    std::printf("inference failed: %d\n", inference_rc);
    return;
}
for (const auto& r : results) {
    // r.code                     : 0 (FACE_OK) means usable; negative values indicate quality/pose/bounds issues, etc.
    // r.x, r.y, r.width, r.height : Face box (pixels in the original image)
    // r.conf                     : Face-detection confidence
    // r.yaw / r.pitch / r.roll   : Head pose (yaw / pitch / roll)
    // r.key_pts                  : Coordinates of 21 landmarks
    // r.feature                  : 512-dimensional face feature vector (for 1:N matching)
}

Input requirements:

Input must be non-empty and use contiguous BGR memory. A cropped ROI cv::Mat is not necessarily contiguous; check isContinuous() and call clone() if needed before passing it to the SDK.
channels must be 3. Convert grayscale images to BGR with cv::cvtColor first.
The SDK performs model preprocessing internally. The caller must still ensure that the image format, channel count, memory continuity, and orientation are correct; do not repeat model normalization yourself.

Output structureFaceResults_t (one per face):

Field	Meaning
code	0 means the face is usable; negative values indicate that a feature could not be extracted due to quality, pose, bounds, or other issues.
x / y / width / height	Face box in original-image pixels
conf	Face-detection confidence
yaw / pitch / roll	Head-pose angles (degrees)
key_pts	21 face-landmark points
feature	This model produces a 512-dimensional face feature; before matching, check that code == FACE_OK and feature is non-empty.

1:N Recognition¶

Use cosine_similarity to compare the current face feature with the database (one feature per person) using cosine similarity. It returns the matched index and score.

// Database: run inference once for each image under facedb/, then use the filename without its extension as the name.
std::pair<int,float> m = engine.cosine_similarity(db_features, face.feature, 0.6f);
// m.first >= 0 : Match at index m.first in the database; m.first == -1 : unknown face (Unknown)
// m.second     : Cosine similarity score (higher means more similar)

The third argument is the match threshold (default 0.6): a face is recognized only if its similarity is above the threshold and it is the top-1 match; otherwise it is Unknown.
At startup, each image in facedb/ is read in turn. For each image, only the first face with code == FACE_OK and a non-empty feature is used; its name is the filename without the extension. Use base images containing exactly one target person to avoid assigning the wrong identity.
When [Register Face] is clicked, the largest usable face is selected. Its name and 512-dimensional feature are atomically saved to facedb/registered/*.json; after a successful save, the entry is added to the in-memory database. The camera image is not saved. On the next launch, base images are loaded first, followed by valid registered entries. Multiple samples for the same name are retained; existing entries are not overwritten. Use an English name or ID where possible because the current OpenCV image text does not support full Chinese names.
Registration JSON files use random filenames and owner-only read/write permissions. Run the program as the same user each time. Back up and protect biometric data in facedb/registered/, and do not commit it to a public repository. Re-register faces after changing the feature model.
A higher threshold is stricter (fewer false matches, more rejections); lower it as appropriate when device or lighting conditions differ significantly.

Release¶

engine.uinit();   // Release detection / landmark / recognition models and authorization objects

Deployment and Run¶

Transfer the Files¶

Run the following from Windows PowerShell on the PC, in the directory containing the SDK package and project. Push the files to /tmp on the device first.

adb devices
adb push "RK_FR_NPU_v1.1/rknn-2.3.2-linux" /tmp/
adb push "face_project" /tmp/

Then open a terminal in the Quectel Pi M2 desktop session and run the following as the regular user. These commands require sudo. If a directory with the same name already exists in the home directory, stop the program and back up the license and face database before updating; otherwise, the new directory could be moved inside the old one.

sudo mv /tmp/rknn-2.3.2-linux /tmp/face_project "$HOME"/
sudo chown -R "$$USER:$$(id -gn)" "$$HOME/rknn-2.3.2-linux" "$$HOME/face_project"
cd "$HOME/face_project" && chmod +x *.sh

Install the Build Environment and Tools¶

Run on the device when it is connected to the network:

sudo apt update
sudo apt install -y build-essential pkg-config qtbase5-dev

Copy Dependencies¶

setup_from_sdk.sh copies headers, shared libraries, models, sample images, and the face database from the SDK package into the corresponding project directories:

cd "$HOME/face_project"
sh setup_from_sdk.sh "$HOME/rknn-2.3.2-linux"   # Omit the argument to use this default path.

Build¶

sh build.sh
#Output: /home/$USER/face_project/face_cam_demo

build.sh invokes make: it uses the system Qt5 for the UI, then links the project-local libfacerecognition.so and bundled OpenCV world library to produce the executable.

Run¶

Run from the device’s desktop session (the Qt window requires a desktop environment):

cd "$HOME/face_project"
./run.sh

Once ./run.sh opens the UI, click Open Camera to start capture. You can also specify a device node based on the output of v4l2-ctl --list-devices, for example ./run.sh /dev/video73; the node number is only an example. If the specified node cannot be opened, the program still attempts to detect a USB capture node automatically.

We recommend launching from a terminal in the device’s desktop session. If using adb shell, switch to the desktop login user and make sure XDG_RUNTIME_DIR, WAYLAND_DISPLAY, or DISPLAY/XAUTHORITY matches the actual desktop session. The script’s display-platform detection cannot guarantee that root or another user can connect to the desktop.

Use the UI¶

The window has three columns: algorithm modules on the left, display and controls in the center, and device status on the right.

Control	Description
[Open Camera]	Open/close the USB camera for real-time detection and 1:N recognition; closing it clears the display.
[Local Image]	Open a file picker and run offline recognition on a selected image.
[Register Face]	Freeze the current frame and prompt for a name; register the largest face so it can be recognized afterward.
[Landmarks]	Toggle display of the 21 face landmarks.
[Full Screen]	Toggle between full-screen and windowed mode.
[Quit]	Close the program.
Keyboard shortcuts	Esc/Q to quit; F for full-screen mode.

The device status panel on the right refreshes once per second:

Metric	Description
Face DB	Current number of faces in the database, including runtime registrations
RAM	Used percentage and used/total memory (MB)
ROM	Root filesystem usage percentage and used/total capacity (GB)
CPU	CPU utilization percentage

Authorization¶

The license file and application configuration must match the device and the scope of the license:

Supply the authorization code through FACE_AUTH_CODE (already exported by run.sh; replace its value with your own code).
FACE_MODULE defaults to 3 (RK3576; see include/authorization.h).
FACE_APP_ID currently defaults to 145, corresponding to CV_SET_APPID.
FACE_LICENSE_FILE defaults to the absolute path $ROOT/face_license.ini inside the project. If no valid local license exists, the SDK attempts to obtain one online. After successful authorization, it saves a device-bound file and prefers that local file on subsequent launches. Keep the file and make sure the runtime user can write to the project directory. Do not distribute authorization codes, license files, or face databases with a public project.
A valid local license may allow authorization to succeed without an online request using the newly supplied code. When testing a new code, distinguish the test from reuse of a local license; do not infer success solely from the file’s existence, its non-empty contents, or the UI opening.

Status queries: get_status(0) returns algorithm initialization status (0 means success); get_status(1) returns authorization status (0 means success; other values are defined in AuthErrCode_e).

Demo Video¶

face-demo

Troubleshooting¶

Issue	Possible cause	Solution
Initialization fails (alg_status != 0)	Model is missing or initialization failed	Copy dependencies first, verify the paths to all three models, and use the SDK return code to investigate.
Authorization fails (auth_status != 0)	License, application ID, device binding, or resource scope does not match	Check FACE_AUTH_CODE, FACE_APP_ID, and FACE_MODULE; contact the licensing provider. Do not change the module number without confirmation.
auth_status=100024	The requested FaceRecognition resource is outside the license scope	Ask the licensing provider to confirm the application and resource entitlements.
Registration cannot be saved or is not restored after restart	Directory is not writable, disk is full, record is corrupt, or the runtime user/model changed	Check facedb/registered/, file ownership, and runtime logs; launch from the same project directory. Re-register after changing the feature model.
Window does not appear	Not running in a desktop session / QT_QPA_PLATFORM does not match	Run in the device’s desktop session, or let run.sh detect Wayland/xcb automatically.
Camera cannot be opened	USB camera is disconnected / specified node is wrong	Automatic mode probes for USB capture nodes. Use v4l2-ctl –list-devices to check, then run ./run.sh /dev/videoN.
Link or runtime error: .so not found	LD_LIBRARY_PATH is not set	Launch with run.sh (which configures it), or export the project lib path manually.
Face is labeled low-quality	A face box was detected, but pose, size, bounds, or quality did not meet the requirements for feature extraction	Face the camera directly and keep the face clear and fully in frame; adjust the detection confidence in set_param if needed.

Support and Contributions¶

For help, submit a technical question to the Quectel official forum, where the Quectel technical support team can assist.

Project repository: https://github.com/Quectel-Pi/demo-edge-algorithms

Issues and pull requests for improvements are welcome.